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INTRODUCTION: 


We  are  developing  new  imaging  techniques  based  on  ultrasound  elastography  for 
monitoring  the  changes  in  size,  shape  and  stiffness  of  tumors  in  patients  receiving 
Neoadjuvant  chemotherapy  (NAC).  NAC  is  often  administered  to  women  with  operable 
stage  II  or  III  breast  cancer  with  an  operable  tumor.  It  may  shrink  the  size  of  the  tumor 
and  enhance  breast  conservation.  Success  of  NAC  depends  on  assessment  of  chemo- 
sensitivity  and  early  detection  of  the  response  of  the  tumor  to  the  administered  drug. 
However,  the  clinical  tools  used  to  monitor  the  response  of  cancer  to  NAC  are  less  than 
ideal  due  to  imprecision,  high  cost,  and  accessibility  issues. 

Ultrasound  systems  on  the  other  hand  are  widely  accessible,  cost-effective,  and  portable. 
Therefore,  an  ultrasound-based  monitoring  system  could  be  used  to  regularly  screen  the 
changes  in  the  tumor  during  the  procedure  and  help  with  timely  detection  of  drug 
resistance.  We  have  been  developing  such  a  system,  and  our  main  achievements  towards 
this  goal  can  be  categorized  into  four  parts. 

First,  we  built  the  interface  to  connect  a  new  3D  ultrasound  probe  ideal  for  elastography 
to  our  ultrasound  machine.  This  involved  both  hardware  and  software  development.  The 
hardware  is  composed  of  probe  wiring  and  connection  to  the  ultrasound  machine,  a  motor 
controller,  and  a  digitizer  to  record  the  position  of  the  probe.  The  software  sends 
commands  to  the  transducer  and  controls  its  motion.  There  are  also  functions  available  to 
read  raw  data  from  the  probe,  save  the  data,  reconstruct  3D  volumes,  and  send  the  data 
over  a  local  network.  This  task  had  been  completed  in  collaboration  with  the  MUSiiC 
laboratory  at  the  Laboratory  for  Computational  Sensing  and  Robotics,  Johns  Hopkins 
University  [1]. 

Second,  we  developed  a  customized  stereo  camera  system  with  structured  light  to 
reconstruct  the  surface  of  the  patient’s  breast.  This  task  was  not  directly  sought  in  the 
original  statement  of  work.  The  3D  surface  reconstruction  system  largely  aids  the 
deformable  registration  and  localization  of  the  tumor  over  time.  The  idea  is  to  reconstruct 
the  3D  surface  of  the  breast  each  time  prior  to  data  collection  from  the  patient.  This 
surface  is  registered  to  the  pre -treatment  CT  scan  of  the  patient  which  will  provide  a 
base-line  for  comparison.  This  system  is  capable  of  mapping  the  surface  with  sub¬ 
millimeter  accuracy.  The  structured  light  in  this  system  compensates  for  the  lack  of 
features  on  the  surface  of  the  skin. 

Third,  we  further  developed  our  tracked  ultrasound  elastography  (TrUE)  to  achieve  better 
accuracy,  increased  robustness  to  noise,  and  higher  image  quality  [2],  We  now  exploit  the 
image  content  in  addition  to  the  tracking  information  to  refine  the  displacement  field.  We 
have  also  incorporated  a  new  strain  estimation  method  that  analytically  solves  for  motion 
field  [3].  Furthermore,  we  fuse  multiple  images  to  increase  the  signal-to-noise  ratio 
(SNR)  of  the  output  image. 

Finally,  we  have  carried  out  a  series  of  experiments  on  a  breast  phantom,  an  elastography 
phantom,  and  pig’s  liver.  The  breast  phantom  was  employed  to  test  our  structured  light 
stereo  system  and  our  tracked  elastography  method.  A  more  controlled  experiment  was 
possible  with  the  elastography  phantom  since  the  exact  properties  of  this  phantom  are 
known.  The  ex -vivo  and  in-vivo  experiment  with  the  pig’s  liver  provides  a  more 
challenging  environment  close  to  clinical  setups  with  the  associated  uncertainties.  The  in- 
vivo  experiment  is  especially  challenging  due  to  the  internal  motions  such  as  blood  flow 


1 


and  breathing.  The  collected  data  from  these  experiments  are  only  partially  analyzed,  and 
complete  results  will  be  published  in  near  future. 

I  have  also  expanded  my  knowledge  about  various  aspects  of  breast  cancer,  medical 
imaging,  and  elastography.  As  a  part  of  my  training,  I  regularly  attended  weekly  meeting, 
seminars,  and  a  journal  club. 


BODY: 

Task  1 :  Breast  cancer  training 

My  clinical  and  engineering  mentors  have  been  very  helpful  by  both  tutoring  me  and 
directing  me  to  the  relevant  articles,  books,  and  online  resources.  In  this  period,  I 
have  been  studying  the  various  subjects  on  breast  cancer  such  as  the  types,  the  risk 
factors,  and  the  epidemiology  of  breast  cancer.  I  also  learned  more  about  treatment 
options  available  for  different  stages  and  types  of  breast  cancer  especially  NAC. 
Through  regular  meetings,  we  discussed  research  issues,  applicability  and  practicality 
of  our  proposed  techniques,  the  problems  that  need  to  be  resolved  and  potential 
solutions.  I  closely  followed  the  advancements  in  ultrasound  and  elastography  by 
attending  a  weekly  journal  club  and  presenting  recent  publications.  I  also  broadened 
my  scientific  knowledge  by  attending  seminars  and  lab  meetings.  Although  the 
subjects  of  these  seminars  may  not  have  been  directly  related  to  my  research,  but  they 
exposed  me  to  new  ideas  and  approaches  that  could  be  useful  in  my  own  research. 
The  topics  included  computer  integrated  diagnoses  and  intervention,  medical  image 
analysis,  image  processing,  ultrasound,  and  physics  of  medical  imaging. 

Task  2:  System  integration  and  registration 

2a.  3D  reconstruction 

3D  reconstruction  refers  to  creating  volumetric  data  from  2D  images  for  which  spatial 
information  is  available.  The  images  may  be  b-mode  ultrasound  or  estimated  strain 
maps.  We  have  implemented  a  C++  code  that  reads  the  input  images  and  their 
corresponding  spatial  transfonnations,  applies  probe  calibration  to  the 
transformations,  and  reconstructs  a  3D  volume.  Two  methods  have  been  implemented 
for  reconstruction.  Either  a  value  for  each  voxel  in  the  volume  is  south  by  finding  the 
nearest  pixels  in  all  images,  or  the  pixels  of  the  images  are  scanned  and  merged  into 
the  nearest  voxel  [4].  Interpolation  is  necessary  for  both  approaches.  We  have  also 
developed  visualization  software  with  tools  for  manipulating  and  reslicing  the 
volumes  using  visualization  toolkit  (VTK)  [5].  These  programs  have  been  integrated 
into  our  data  collection  and  synchronization  software. 

2b.  3D  probe  interface  and  volume  stitching 

We  have  a  new  high-frequency  linear  transducer  capable  of  moving  its  elements  in 
precise  linear  steps  (micrometer  resolution)  to  construct  a  cubic  ultrasound  volume 
(Figure  1).  This  transducer  has  multiple  advantages  over  the  traditional  wobbler  and 
the  linear  array  probes.  The  linear  motion  of  this  transducer  makes  it  suitable  for 


2 


volumetric  elastography  since  the  spacing  between  the  lines  remains  constant,  and 
there  are  fewer  variations  in  the  size  of  the  resolution  cell.  Also,  uniform  force  can  be 
applied  to  the  tissue  due  to  the  flat  surface  of  the  probe.  This  is  an  important  feature 
for  elastography  which  helps  in  constructing  a  unifonn  image.  The  volume  is  also 
large  compared  to  linear  arrays. 


Figure  1 :  The  3D  ultrasound  transducer  and  our  interface  board. 

We  built  an  interface  for  this  probe  shown  in  Figure  1.  This  interface  is  required  for 
acquiring  ultrasound  data  from  the  probe  and  consists  of  both  hardware  and  software 
components.  The  hardware  has  three  parts.  There  is  a  connector  to  attach  the  probe  to 
the  ultrasound  machine.  The  wiring  for  the  connector  is  made  possible  by  6  PCB 
boards  each  of  which  has  6  metal  layers.  Three  of  these  layers  are  used  for  shielding 
and  suppressing  the  noise.  The  motor  controller  has  a  separate  board  and  controls  the 
step  motion  of  the  elements  inside  the  probe.  The  last  component  is  a  digitizer  that 
converts  the  position  feedback  from  the  controller  unit  into  digital  signals  (Figure  2). 
The  controller  and  the  digitizer  are  connected  to  the  ultrasound  machine  via  USB  port. 
The  power  for  the  controller  is  provided  by  a  separate  power  supply. 


Figure  2:  The  hardware  interface  for  the  3D  transducer 


The  software  component  detects  the  probe  and  controls  the  imaging  parameters  as 
well  as  the  motion  of  the  motor  of  the  transducer.  By  sending  commands  to  the 
controller,  it  adjusts  the  speed,  step  size,  and  direction  of  the  motor.  This  software  is  a 
part  of  a  toolkit  developed  in  MUSiiC  lab  [1]  referred  to  as  MUSiiC-Toolkit  [6].  We 
also  have  a  protocol  to  send  the  images  (both  2D  and  3D)  over  a  local  network  to  a 
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third  party  computer  for  further  processing  based  on  openlGTlink  [7].  This  will 
facilitate  extended  computational  power  for  real-time  processing. 

2c.  Deformable  registration 

We  have  revised  the  original  plan  for  registration  and  localization  of  the  tumors.  In 
the  original  plan,  we  intended  to  directly  register  the  3D  scans  of  the  tumor  together, 
which  is  a  difficult  task  and  may  require  additional  supervision  for  failure  detection. 
It  also  does  not  capture  the  shifts  in  tumor  position  with  respect  to  the  breast  anatomy. 
In  the  new  plan,  we  employ  a  stereo  camera  system  to  capture  the  3D  surface  of  the 
breast.  This  surface  is  then  registered  to  the  surface  extracted  from  the  pre-treatment 
CT  scan.  We  will  use  a  newly  introduced  point  set  registration  method  called 
“coherent  point  drift”  [8]  for  deformable  registration.  The  advantage  is  that  the  CT 
will  provide  a  base-line  for  all  ultrasound  scans.  The  registration  will  be  more  reliable 
and  it  will  be  easier  to  track  the  changes  in  shape,  size,  position,  and  stiffness  of  the 
tumor  since  the  images  taken  during  the  course  of  treatment  are  aligned  with  respect 
to  the  patient’s  breast. 

To  this  end,  we  have  developed  a  customized  stereo  system  to  reconstruct  the  3D 
surface  of  breast.  This  system  is  specifically  designed  for  breast  imaging  and  consists 
of  two  cameras  and  a  projector  (see  Figure  3). 


Figure  3:  Structured  light  stereo  system 


This  device  is  mounted  on  a  tripod  facing  down  toward  the  breast.  The  projector 
projects  several  light  patterns  on  the  breast  while  the  two  cameras  records  the  images 
of  the  breast  (Figure  4  left).  The  structure  light  creates  virtual  features  on  the 
featureless  surface  of  the  breast.  The  pictures  from  individual  patterns  are  stacked 
together  creating  a  vector  of  several  RGB  values  for  each  pixel.  Having  the  camera 
calibration,  the  corresponding  points  in  the  images  are  triangulated  to  solve  for  the 
depth  map  (Figure  4  right).  For  this  purpose,  we  have  developed  a  C++  code  as  a 
“mex”  function  that  can  be  called  within  MATLAB.  This  code  is  optimized  for  speed 
and  can  reliably  generate  3D  depth  maps  since  several  images  with  different  light 
patterns  are  used.  The  patterns  are  specifically  designed  for  this  application.  With  the 
current  implementation,  only  a  few  seconds  are  required  to  reconstruct  the  3D  map  of 
the  surface  with  sub-millimeter  accuracy. 


4 


Task  3.  Development  of  2D  and  3D  TrUE 

3a.  Fusion  of  DP-based  USEI  and  TrUE  framework 

Tracked  ultrasound  elastography  (TrUE)  is  a  frame  selection  technique  which  seaches 
for  the  best  frame  pairs  that  could  be  used  for  elastography  [2],  In  TrUE,  we  exploited 
the  tracking  data  to  enhance  the  quality  of  the  elasticity  images  in  two  ways.  First, 
having  the  tracking  information,  multiple  pairs  of  images  containing  optimal  hand 
motion  are  selected.  The  optimum  value  for  lateral  and  out-of-plane  motions  is  zero, 
and  the  optimum  axial  motion  is  selected  by  the  user.  Second,  the  strain  images 
obtained  from  these  pairs  are  fused  together  based  on  the  location  of  each  strain 
image  to  improve  image  quality.  We  now  exploit  the  image  content  in  addition  to  the 
tracking  information  to  refine  the  displacement  field.  The  details  of  this  method  can 
be  found  in  the  attached  appendix,  Section  2. 

In  original  implementation  of  TrUE,  we  used  a  normalized  cross  correlation  (NCC) 
based  technique  to  estimate  the  displacement  map.  NCC  is  slow  and  prone  to  errors 
due  to  decorrelation.  In  our  new  implementation,  we  have  incorporated  a  new  strain 
estimation  method  that  is  fast  and  shown  to  be  robust  to  noise  and  outliers  [3].  Our 
initial  evaluations  are  very  promising.  A  complete  analysis  of  the  performance  of  this 
technique  is  underway. 

Task  4.  Clinical  data  collection,  analysis,  and  validation 

4a.  Breast  phantom  evaluation 

We  carried  out  experiments  on  breast  phantoms  and  pig  liver  to  evaluate  our 
developed  imaging  techniques.  The  results  of  these  experiments  are  reported  in  the 
appendix,  Section  3.  We  are  processing  the  in-vivo  data  taken  from  pig  liver.  The 
advantage  of  experimenting  with  live  tissue  is  that  the  internal  motions  due  to 
breathing  and  blood  flow  are  not  omitted.  These  experiments  also  help  us  detect 
possible  glitches  and  improve  our  system  for  fast  data  collection.  Based  on  these 
experiments  we  will  be  working  on  a  data  acquisition  protocol  with  breast  cancer 
patients. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


1.  Software  development  for  data  collection,  volume  reconstruction,  and 
visualization. 

2.  Interfacing  a  new  3D  ultrasound  transducer  for  elastography. 

3.  Building  a  stereo  camera  system  with  structured  light  optimized  for  breast 
imaging. 

4.  Enhancing  the  performance  our  frame  selection  technique  by  using  image  content, 
fusing  multiple  images  together,  and  incorporating  an  advanced  displacement 
estimation  technique. 

5.  Evaluating  the  developed  techniques  on  breast  phantoms  and  pig’s  liver. 

REPORTABLE  OUTCOMES 

•  Pezhman  Foroughi,  Hassan  Rivaz,  Ioana  N.  Fleming,  Gregory  D.  Hager,  and 
Emad  M.  Boctor,  “Tracked  ultrasound  elastography  (TrUE)”  Medical  image 
computing  and  computer-assisted  intervention,  Part  II,  pp.  9 — 16,  2010 


CONCLUSION 

The  proposed  monitoring  system  for  NAC  is  aimed  at  personalizing  the  treatment  for 
patients  undergoing  NAC  procedure  by  early  detection  of  drug  resistance  and  assessment 
of  chemo-sensitivity.  In  this  period,  we  have  been  developing  the  components  of  our 
monitoring  system  and  creating  the  necessary  testbed  for  evaluation  of  the  effectiveness 
of  it.  The  new  3D  probe  is  now  ready  for  data  acquisition  and  the  structured  light  stereo 
system  can  create  high  resolution  3D  maps  of  the  surface  of  the  breast.  This  map  will  be 
used  for  registration  of  datasets  taken  in  different  time  intervals.  The  preliminary  results 
of  our  tracked  ultrasound  elastography  method  show  that  it  can  produce  consistent  and 
high-quality  strain  images.  In  the  next  step,  we  will  be  integrating  all  these  components 
into  one  system.  We  will  also  be  conducting  more  experiments  and  analysis  including  in- 
vivo  animal  experiments  to  test  the  resistance  of  TrUE  to  internal  motions  such  as  the 
ones  induced  by  heart-beat  and  respiration.  Finally,  we  will  move  toward  a  complete 
clinical  trial  by  first  seeking  IRB  and  other  necessary  approvals.  Although  our  specific 
target  is  breast  cancer,  there  are  several  other  clinical  applications  which  may  benefit 
from  the  techniques  developed  in  this  work. 
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Abstract.  This  paper  presents  a  robust  framework  for  freehand  ultra¬ 
sound  elastography  to  cope  with  uncertainties  of  freehand  palpation  us¬ 
ing  the  information  from  an  external  tracker.  In  order  to  improve  the 
quality  of  the  elasticity  images,  the  proposed  method  selects  a  few  im¬ 
age  pairs  such  that  in  each  pair  the  lateral  and  out-of-plane  motions  are 
minimized.  It  controls  the  strain  rate  by  choosing  the  axial  motion  to  be 
close  to  a  given  optimum  value.  The  tracking  data  also  enables  fusing 
multiple  strain  images  that  are  taken  roughly  from  the  same  location. 
This  method  can  be  adopted  for  various  trackers  and  strain  estimation 
algorithms.  In  this  work,  we  show  the  results  for  two  tracking  systems 
of  electromagnetic  (EM)  and  optical  tracker.  Using  phantom  and  ex-vivo 
animal  experiments,  we  show  that  the  proposed  techniques  significantly 
improve  the  elasticity  images  and  reduce  the  dependency  to  the  hand 
motion  of  user. 

Keywords:  Ultrasound,  Elastography,  Elasticity,  Tracking,  Strain. 


1  Introduction 

Ultrasound  elastography  is  an  emerging  medical  imaging  modality  which  in¬ 
volves  imaging  the  mechanical  properties  of  tissue  and  has  numerous  clinical 
applications.  Among  many  variations  of  ultrasound  elastography  [1],  our  work 
focuses  on  real-time  static  elastography,  a  well-known  technique  that  applies 
quasi-static  compression  of  tissue  and  simultaneously  images  it  with  ultrasound. 
Within  many  techniques  proposed  for  static  elastography,  we  focus  on  freehand 
palpation  elasticity  imaging  which  involves  deforming  the  tissue  by  simply  press¬ 
ing  the  ultrasound  probe  against  it.  Freehand  ultrasound  elastography  has  shown 
great  potential  in  clinical  applications  especially  for  diagnosis  and  screening  of 
breast  lesions  [2].  The  application  of  elastography  is  not  limited  to  breast,  and 
other  applications  such  as  diagnosis  of  prostate  cancer,  monitoring  ablation  and 
deep  vein  thrombosis  have  also  been  studied. 

Despite  the  reports  on  success  of  elastography,  yet  it  has  not  become  a  part  of 
any  routine  clinical  application.  The  main  reason  is  that  elastography  is  highly 
qualitative  and  user-dependent.  The  best  result  is  achieved  when  the  user  com¬ 
presses  and  decompresses  the  tissue  uniformly  in  the  axial  direction  with  the 
proper  hand  motion.  It  is  difficult  to  control  the  compression  rate  as  it  is  gov¬ 
erned  by  the  hand  motion  and  the  frame  rate  of  RF  data.  Also,  small  lateral 
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or  out-of-plane  motions  can  compromise  the  quality  of  images.  However,  it  is 
difficult  to  induce  pure  axial  motion  with  freehand  compression.  Sophisticated 
algorithms  can  only  partially  address  the  problem  by  compensating  for  in-plane 
motions  and  applying  smoothness  constraints.  The  images  are  also  hard  to  in¬ 
terpret,  and  artifacts  -caused  by  failure  of  the  strain  estimation  algorithm  or 
poor  hand  motion-  may  be  mistaken  for  lesions  inside  the  soft  tissue.  Devel¬ 
oping  an  elastography  technique  that  is  not  affected  by  poor  hand  motion  and 
other  sources  of  signal  decorrelation  will  pave  the  way  for  wide-spread  clinical 
use  of  elastography. 

To  improve  the  reliability,  quality  metrics  such  as  persistence  in  strain  images 
have  been  developed  [3,4].  This  quality  indicator  is  calculated  for  each  image 
and  provided  to  the  user  as  feedback.  Persistence  is  also  used  to  merge  multiple 
elasticity  images  together  [3],  To  measure  the  persistence,  strain  is  computed 
for  two  pairs  of  echo  frames,  and  the  resulting  images  are  correlated.  Although 
these  techniques  offers  a  major  advantage,  there  remains  several  limitations. 
First,  the  strain  has  to  be  estimated  before  the  calculation  of  the  quality  metric. 
With  typical  ultrasound  settings,  the  frame  rate  can  reach  more  than  30  Hz.  For 
subsequent  frames,  an  efficient  implementation  of  this  image-based  metric  might 
cope  with  this  rate.  Nonetheless,  the  task  will  be  extremely  difficult  to  try  all 
the  combinations  in  a  series  of  frames.  Moreover,  the  quality  metric  will  not  be 
able  to  provide  feedback  to  the  user  whether  he/she  should  adjust  the  palpation 
in  certain  direction.  Also,  there  would  be  minimal  control  over  the  strain  rate. 

The  ultrasound  probe  is  often  tracked  in  navigation/guidance  systems  to  pro¬ 
vide  spatial  information,  to  form  freehand  3D  ultrasound,  or  to  facilitate  multi- 
modality  registration.  In  this  work,  we  exploit  the  tracking  data  to  enhance  the 
quality  of  the  elasticity  images.  We  use  the  tracking  data  to  select  multiple  im¬ 
age  pairs  that  contain  the  optimum  deformation  for  the  elastography  algorithm. 
The  optimum  value  for  lateral  and  out-of-plane  motions  is  zero,  and  the  optimum 
axial  motion  is  determined  by  the  specific  elastography  algorithm  used,  which 
is  Normalized  Cross- Correlation  (NCC)  in  this  work.  Next,  we  fuse  the  strain 
images  obtained  from  the  multiple  image  pairs  together  based  on  the  location 
of  each  strain  image  to  improve  image  quality.  We  assume  that  the  ultrasound 
data  is  2D.  Nonetheless  similar  techniques  proposed  here  could  be  extended  to 
3D  ultrasound. 

2  Methodology 

Consider  a  sequence  of  RF  data  collected  during  the  palpation  of  tissue  using  a 
tracked  transducer.  We  have  previously  shown  that  it  is  possible  to  synchronize 
the  RF  frames  with  the  tracking  information  relying  only  on  the  same  data 
collected  during  palpation  [5].  From  synchronization,  the  tracking  information 
is  interpolated  at  the  incident  time  of  each  frame.  The  input  to  our  algorithm  is 
then  a  series  of  RF  frames  along  with  their  corresponding  transformation. 

First,  we  need  to  define  a  distance  function  between  two  frames  of  RF  data. 
For  this  purpose,  we  use  a  model  of  image  decorrelation  in  presence  of  out-of- 
plane  and  lateral  motion.  RF  signal  is  often  modeled  as  the  collective  response 
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of  scatterers  randomly  distributed  within  the  resolution  cell  of  the  ultrasound 
[6,7].  Each  scatterer  is  assumed  to  have  an  amplitude  governed  by  the  shape  of 
the  resolution  cell  and  a  phase  which  is  distributed  from  0  to  7r  uniformly  at 
random.  Considering  a  Gaussian  shape  for  the  resolution  cell  Prager  et.  al  [8] 
calculated  the  correlation  as  a  function  of  out-of-plane  motion  to  be  exp(  —  j^?). 
8  and  a  denote  the  displacement  and  the  width  of  the  resolution  cell  respectively. 
Although  this  function  is  only  valid  for  fully  developed  speckle,  it  provides  a  con¬ 
venient  estimate  of  correlation.  It  should  be  noted  that  in  [8],  the  displacement  is 
estimated  from  correlation,  whereas  here,  we  intend  to  define  an  energy  function 
based  on  displacement.  Extending  this  formula  to  both  out-of-plane  and  lateral 
displacements,  we  define  our  energy  function,  E(x,z),  as  follows: 

E(DX,  Dz)  =  exp(—Kx  ■  D\  -  Kz  ■  D2Z),  (1) 

where  Dx  and  Dz  represent  the  displacement  in  out-of-plane  and  lateral  direc¬ 
tions.  E  does  not  depend  on  axial  motion  (  Dy )  since  displacement  in  axial 
direction  is  necessary  for  strain  estimation.  Kx  and  Kz  determine  the  sensi¬ 
tivity  to  a  certain  direction.  In  order  to  be  able  to  use  this  function,  we  need  a 
component- wise  metric  representing  the  distance  of  two  frames  given  their  homo¬ 
geneous  transformations.  The  first  step  is  to  compute  the  relative  transformation 
between  them.  Suppose  a  =  [ ax  ay  az]T  is  the  axis-angle  representation  of  the 
relative  rotation,  and  t  =  [tx  ty  tz]T  is  the  relative  translation.  Assuming  a  small 
rotation,  the  relative  displacement  of  a  point,  P  =  [xy  0]T,  will  b  e  d  =  ax  P  +  t. 
We  then  define  the  distance  vector  of  two  frames,  D  =  [ Dx  Dy  DZ]T ,  as  the  RMS 
of  the  components  of  d  for  all  the  points  in  the  region  of  interest  (ROI): 
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where  sqrt{.}  returns  the  root.  Here,  ROI  is  assumed  to  be  rectangular  and 
determined  by  x±,  x2,  yi,  and  y2.  The  vector  D  provides  a  measure  of  distance 
for  each  direction  separately.  We  use  this  vector  in  Equation  (1)  which  gives  us 
an  estimate  of  “pseudo-correlation”  over  the  ROI. 

The  data  goes  through  four  stages  of  processing  to  create  a  single  high-quality 
strain  image.  In  the  first  step,  few  images  are  selected  from  the  data  series 
that  are  approximately  collected  from  one  cross-section  of  tissue  with  minimal 
lateral  and  out-of-plane  motion.  To  this  end,  the  energy  function  of  each  frame 
is  computed  with  respect  to  all  other  frames  in  the  sequence.  Then,  the  total 
energy  is  found  for  each  frame  as  the  sum  of  the  energies  of  the  M  closest  frames, 
where  closeness  implies  higher  energy,  and  M  is  the  maximum  number  of  frames 
to  be  selected.  Then,  the  frame  with  the  highest  total  energy  (the  center  frame) 
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is  identified,  and  the  M  closest  frames  to  the  center  frame  including  itself  are 
selected.  Additionally,  the  frames  that  have  E  of  less  than  0.5  with  respect  to  the 
center  frame  are  disqualified.  This  is  applied  to  ensure  lower  number  of  frames 
are  chosen  when  M  frames  from  one  cross-section  are  not  available. 

In  the  next  stage,  the  program  evaluates  all  possible  combination  of  frame 
pairs  for  elastography.  For  M  frames,  there  will  be  (^f)  =  M(M  —  l)/2  pair 
combinations  which  will  be  compared  using  a  slightly  modified  version  of  E. 
Since  the  pairs  are  directly  compared,  it  suffices  to  minimize  the  exponent  of 
Equation  (1)  in  order  to  maximize  E.  We  also  add  a  term  for  axial  motion  that 
penalizes  compressions  that  are  higher  than  an  optimum  compression  value,  topt. 
Hence,  a  “cost  function”,  Cl,  is  defined  as  follows: 

Cl(D),  K,.Dl+K„.Dy2+K,.Dl,  D, -  { £«  ~  (3) 

where  topt  implies  the  optimal  strain,  which  can  be  theoretically  defined  as  de¬ 
scribed  in  [9] .  Here,  topt  is  set  depending  on  the  robustness  of  the  elasticity 
estimation  algorithm.  Its  value  might  be  within  the  range  of  the  resolution  of 
the  tracker.  Therefore,  at  this  stage  we  do  not  assign  a  penalty  for  the  com¬ 
pressions  less  than  topt.  If  the  compression  is  close  to  zero,  the  contrast  of  the 
reconstructed  image  degrades.  The  program  filters  the  pairs  with  low  compres¬ 
sion  in  the  next  stage  using  image  content.  Similar  to  the  first  part,  a  maximum 
number  of  frames  with  lowest  cost  are  selected  provided  that  the  cost  is  lower 
than  a  threshold.  The  threshold  is  not  strict  to  ensure  acceptable  pairs  are  not 
filtered. 

The  final  pairs  are  selected  by  recovering  the  global  lateral  motion  and  com¬ 
pression  by  matching  the  two  RF  frames  in  each  pair.  The  tracking  information 
is  used  to  initialize  the  search.  For  instance,  the  search  range  for  compression  is 
set  to  be  from  zero  to  the  tracker  reading  in  axial  direction  padded  in  both  sides 
with  the  maximum  error  of  the  tracker.  Given  two  frame  I\  and  I2,  the  amount 
of  lateral  motion  a,  and  compression,  6,  is  found  by  solving  cross-correlation: 


arg  max  E  h{x,y)  ■  h{x  +  a,  by)  +  h(x  -  a,  -by)  ■  I2(x,y)  >  .  (4) 

a,b  [x,ySG  J 

The  RF  data  is  normalized  with  standard  variation  and  assumed  to  have  zero 
mean.  We  employ  two  tricks  which  extensively  increases  the  speed  of  search. 
First,  we  do  not  match  the  entire  image  to  solve  for  these  parameters.  Instead, 
only  pixels  on  a  grid,  G,  are  used  as  described  by  Equation  (4).  The  two  terms  of 
Equation  (4)  ensures  that  the  search  remains  reciprocal,  which  means  switching 
the  images  only  affects  the  sign  of  a  and  b.  Second,  a  is  recovered  by  matching 
only  the  top  part  of  the  two  images  while  b  is  fixed  to  one.  The  reason  is  that 
the  displacement  due  to  compression  is  minimal  in  that  region. 

Having  the  global  motions,  the  cost  function  is  modified  to  penalize  very  low 
compressions: 
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C2(D)  =  Kx  ■  Dx2  +  Ky  ■  topt|3  +  Kz  •  D\ ,  (5) 

Dy  +  C 

where  Dx  and  Dy  are  the  global  motions  from  Equation  (4)  converted  to  mm.  c 
is  a  small  number  that  limits  the  cost  of  zero  compression.  Finally,  the  pairs  with 
the  lowest  cost  are  selected  until  a  maximum  number  of  frame  pairs  is  reached 
or  the  minimum  cost  grows  higher  than  the  average  cost. 

The  last  step  involves  computing  the  strain  for  all  the  selected  frame  pairs. 
We  have  implemented  normalized  cross-correlation  (NCC)  [10]  to  recover  the 
displacements  and  least  squares  estimation  to  calculate  the  strain.  Before  calcu¬ 
lating  strain,  the  global  lateral  motion  and  compression  from  the  previous  step 
are  compensated  in  one  image  using  cubic  interpolation.  This  is  known  to  reduce 
the  error  of  strain  estimation  [11].  The  final  strain  image,  Sfinai  is  the  weighted 
average  of  all  the  strains: 


S 


final 


J2i=l  wi  •  S1  _  f  1^7.  Pi  >  °-7 
’  \  0,  otherwise 


(6) 


where  pi  is  the  correlation  coefficient  for  the  *th  pair  after  applying  the  dis¬ 
placements,  and  m  is  the  number  of  pairs.  Fusing  the  strains  in  this  fashion  is 
acceptable  since  the  algorithm  only  allows  for  compressions  that  are  close  to  a 
predetermined  amount  optimal  for  strain  estimation. 


3  Experiments  and  Results 

We  acquired  ultrasound  data  using  a  SONOLINE  Antares™  ultrasound  system 
(Siemens  Medical  Solutions  USA,  Inc.)  with  a  high-frequency  ultrasound  trans¬ 
ducer  (VF10-5)  at  center  frequency  of  6-8  MHz.  We  accessed  RF  through  the  Axius 
Direct 1 M  Ultrasound  Research  Interface  provided  by  Siemens.  Our  custom  data 
acquisition  program  was  connected  to  this  interface  to  send  the  command  for  cap¬ 
turing  RF  data.  At  the  same  time,  the  program  collected  tracking  information  from 
either  a  “Polaris”  optical  tracker  (Northern  Digital  Inc.,  Waterloo,  Canada)  with 
passive  markers  or  the  “medSAFE”  EM  tracker  (Ascension  Tech.  Corp.). 

RF  data  and  tracking  information  was  captured  from  a  breast  phantom  con¬ 
taining  a  harder  lesion  (CIRS  elastography  phantom,  Norfolk,  VA)  and  ex-vivo 
pig  liver.  Alginate  was  injected  to  the  liver  to  mark  a  part  of  liver,  and  then, 
that  area  was  ablated.  The  users  were  asked  to  palpate  the  tissue  over  the  hard 
lesion  in  the  breast  phantom  and  the  ablated  lesion  in  the  pig  liver  while  data 
was  being  collected.  Between  100  to  138  RF  frames  were  acquired  with  the  rate 
of  about  30  frames  per  second. 

The  first  set  of  data  was  captured  by  an  experienced  user  from  the  breast 
phantom.  Figure  1(a)  shows  the  translation  components  of  hand  motion  with 
respect  to  the  first  frame.  The  axial  motion  is  dominant  and  there  is  only  a 
gradual  drift  in  the  lateral  and  elevational  directions.  Figure  1(b)  depicts  the 
high-quality  strain  image  resulting  from  the  TrUE  algorithm. 
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Hand  Motion 


(b)  Strain  image  (TrUE) 


Fig.  1.  (a)  shows  the  translation  of  probe  w.r.t.  the  first  image.  Proper  hand  motion 
is  applied  as  the  axial  compression  is  dominant,  (b)  is  the  output  of  our  proposed 
algorithm. 


Hand  Motion 


Hand  Motion 


(a)  Poor  palpation  with  EM  tacking 


(b)  Poor  palpation  with  optical  tracking 


(c)  Consecutive  vs.  TrUE 


Fig.  2.  Two  cases  of  improper  motions  are  shown  where  the  hand  motion  suffers  from 
large  lateral  and  elevational  components  evident  in  relative  translations.  The  results 
of  case  1  with  EM  tacker  is  shown  on  the  left  column,  and  the  results  of  case  2  with 
optical  tracker  is  shown  on  the  right  column. 
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Applying  a  compression  similar  to  the  one  shown  in  Figure  1(a)  is  a  difficult 
task  for  novice  or  even  intermediate  users.  This  is  especially  the  case  where  axial 
compression  does  not  translate  into  a  simple  up  and  down  motion.  Ultrasound  gel 
creates  a  slippery  surface  that  makes  the  palpation  prone  to  out-of-planc  motion. 
Two  case  are  shown  in  Figure  2,  where  one  is  tracked  with  the  EM  tracker  and 
the  other  one  with  the  optical  tracker.  In  Figure  2(a)  the  hand  motion  contains  a 
large  amount  of  out-of-plane  motion,  whereas,  in  Figure  2(b),  the  user  has  moved 
the  probe  laterally.  In  both  cases,  the  TrUE  algorithm  generates  reliable  results. 
Figures  2  (c)  and  (d)  show  the  contrast-to-noise  ratio  (CNR)  and  signal-to-noise 
ratio  (SNR)  of  the  strain  image.  The  CNR  and  SNR  value  are  computed  from: 


CNR 


2 (at  -  st)2 

+  of 


SNR 


s 

5 

a 


(7) 


where  s  and  a  denote  the  mean  and  standard  deviation  of  intensities.  The  t  or 
b  subscripts  show  that  the  computation  is  only  for  the  target  or  the  background 
region,  respectively.  The  SNR  and  CNR  for  computing  the  strain  from  consecu¬ 
tive  frames  (the  dashed  curve)  is  compared  to  the  SNR  and  CNR  of  the  strain 
image  from  the  proposed  method  (solid  line).  Using  consecutive  frames  is  the 
standard  method  of  elastography  in  ultrasound  machines.  Almost  in  all  cases 
the  TrUE  algorithm  outperforms  the  consecutive  frames  by  a  large  margin. 

Although  the  SNR  and  CNR  provide  quantitative  measures  to  compare  the 
strain  images,  they  do  not  directly  reflect  the  visual  quality  of  strain.  In  Figure  3, 
we  show  results  of  elastography  using  our  frame  selection  technique  as  well  as 
four  other  strain  images  calculated  from  consecutive  frames.  The  Figure  shows 
the  effects  of  improper  compression  in  consecutive  frames  in  the  strain  image. 
At  the  same  time  our  algorithm  provides  a  single  reliable  strain. 


(a)  B-mode  (b)  TrUE  (c)  1&2 


'  it  i  ‘'Mi 

(d)  2&3  (e)  3&4  (f)  4&5 


Fig.  3.  Comparison  of  the  strain  from  TrUE  vs.  consecutive  frames  for  ex-vivo  pig  liver 
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4  Discussion 

We  presented  a  method  of  ultrasound  elastography  which  is  robust  to  the  quality 
of  the  hand  motion  of  the  user.  Using  the  information  from  an  external  tracker, 
it  automatically  selects  multiple  frame  pairs  with  a  specific  compression  and 
minimal  undesired  motions.  Our  approach  does  not  take  into  account  the  tissue 
motion  from  other  sources  such  as  breathing  or  patient  motion.  However,  these 
types  of  motions  are  not  normally  problematic  since  they  occur  with  a  slower 
pace  compared  to  hand  motion. 

Our  experiments  shows  that  even  when  the  transducer  has  severe  lateral  or 
out-of-plane  motions,  the  algorithm  still  manages  to  produce  good  results.  The 
multi-stage  frame  selection  and  careful  image  fusion  makes  the  TrUE  method 
less  sensitive  to  tacker  accuracy  and  robust  to  strain  estimation  failures. 

We  are  planning  to  use  the  proposed  method  in  a  breast  cancer  study.  For  this 
purpose,  we  will  be  implementing  our  MATLAB  code  in  C.  The  strain  estimation 
which  is  still  the  bottleneck  of  our  approach  will  be  executed  in  GPU  allowing 
for  the  use  of  sophisticated  algorithms. 
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